Skip to content

Use to_numpy in ImplicitToExplicitIndexingAdapter.__array__ method#11381

Merged
dcherian merged 5 commits into
pydata:mainfrom
weiji14:fix-cupy2
Jun 17, 2026
Merged

Use to_numpy in ImplicitToExplicitIndexingAdapter.__array__ method#11381
dcherian merged 5 commits into
pydata:mainfrom
weiji14:fix-cupy2

Conversation

@weiji14

@weiji14 weiji14 commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes TypeError: Implicit conversion to a NumPy array is not allowed. Please use `.get()` to construct a NumPy array explicitly when using a backend engine that loads data onto CuPy, but indexes should stay as NumPy (following convention at xarray-contrib/cupy-xarray#90)

Full traceback
TypeError: Implicit conversion to a NumPy array is not allowed. Please use `.get()` to construct a NumPy array explicitly.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[10], line 20, in test_xarray_backend_open_mfdataset()
     13         engine="cog3pio",
     14         concat_dim="band",
     15         combine="nested",
     16     )
---> 17     ds.raster.load()
     18     assert ds.sizes == {"band": 2, "y": 183, "x": 183}
     19     assert ds.x.min() == 700260.0
     20     assert ds.x.max() == 809460.0
File ~/mambaforge/envs/cupy-xarray/lib/python3.14/site-packages/xarray/core/dataarray.py:1173, in DataArray.load(self, **kwargs)
   1143 def load(self, **kwargs) -> Self:
   1144     """Trigger loading data into memory and return this dataarray.
   1145
   1146     Data will be computed and/or loaded from disk or a remote source.
   (...)   1171     Variable.load
   1172     """
-> 1173     ds = self._to_temp_dataset().load(**kwargs)
   1174     new = self._from_temp_dataset(ds)
   1175     self._variable = new._variable
File ~/mambaforge/envs/cupy-xarray/lib/python3.14/site-packages/xarray/core/dataset.py:569, in Dataset.load(self, **kwargs)
    565         if chunked_data:
    566             chunkmanager = get_chunked_array_type(*chunked_data.values())
    567
    568             # evaluate all the chunked arrays simultaneously
--> 569             evaluated_data: tuple[np.ndarray[Any, Any], ...] = chunkmanager.compute(
    570                 *chunked_data.values(), **kwargs
    571             )
    572
File ~/mambaforge/envs/cupy-xarray/lib/python3.14/site-packages/xarray/namedarray/daskmanager.py:85, in DaskManager.compute(self
, *data, **kwargs)
     80 def compute(
     81     self, *data: Any, **kwargs: Any
     82 ) -> tuple[np.ndarray[Any, _DType_co], ...]:
     83     from dask.array import compute
---> 85     return compute(*data, **kwargs)
File ~/mambaforge/envs/cupy-xarray/lib/python3.14/site-packages/dask/base.py:685, in compute(traverse, optimize_graph, scheduler
, get, *args, **kwargs)
    682     expr = expr.optimize()
    683     keys = list(flatten(expr.__dask_keys__()))
--> 685     results = schedule(expr, keys, **kwargs)
    687 return repack(results)
File ~/mambaforge/envs/cupy-xarray/lib/python3.14/site-packages/xarray/core/indexing.py:691, in ImplicitToExplicitIndexingAdapte
r.__array__(self, dtype, copy)
    687 def __array__(
    688     self, dtype: DTypeLike | None = None, /, *, copy: bool | None = None
    689 ) -> np.ndarray:
    690     if Version(np.__version__) >= Version("2.0.0"):
--> 691         return np.asarray(self.get_duck_array(), dtype=dtype, copy=copy)
    692     else:
    693         return np.asarray(self.get_duck_array(), dtype=dtype)
File cupy/_core/core.pyx:1760, in cupy._core.core._ndarray_base.__array__()
-> 1760 'Could not get source, probably due dynamically evaluated source code.'
TypeError: Implicit conversion to a NumPy array is not allowed. Please use `.get()` to construct a NumPy array explicitly.

Somewhat similar to #10078. Helps with some cupy-xarray work that is making TIFF to GPU possible at xarray-contrib/cupy-xarray#81 (comment) (though not fully working yet 🙂).

Checklist

  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

AI Disclosure

Fixes `TypeError: Implicit conversion to a NumPy array is not allowed. Please use `.get()` to construct a NumPy array explicitly` when using a backend engine that loads data onto CuPy, but indexes should stay as NumPy.
@weiji14 weiji14 marked this pull request as ready for review June 12, 2026 06:22
@weiji14

weiji14 commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Should probably add a test for this somehow.

@weiji14 weiji14 marked this pull request as draft June 12, 2026 08:23
@github-actions github-actions Bot added the topic-NamedArray Lightweight version of Variable label Jun 12, 2026
Ensure that instance of ImplicitToExplicitIndexingAdapter's __array__ method works on duck arrays, tested by passing it through np.asarray.

@weiji14 weiji14 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't merge yet, as I don't think the current patch is correct (even though the new test passes).

Edit: should be ok now after 0c405d9.

Comment thread xarray/core/indexing.py Outdated
So that the dtype and copy arguments are handled by np.asarray.
@weiji14 weiji14 marked this pull request as ready for review June 13, 2026 05:28
@weiji14 weiji14 requested a review from dcherian June 17, 2026 10:49
@dcherian dcherian merged commit fb20c68 into pydata:main Jun 17, 2026
43 checks passed
@weiji14 weiji14 deleted the fix-cupy2 branch June 17, 2026 19:07
weiji14 added a commit to xarray-contrib/cupy-xarray that referenced this pull request Jun 18, 2026
Use pydata/xarray#11381 to fix `TypeError: Implicit conversion to a NumPy array is not allowed. Please use `.get()` to construct a NumPy array explicitly`. However, the dask-backed arrays gets loaded into NumPy instead of CuPy, so still need some more work upstream.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic-indexing topic-NamedArray Lightweight version of Variable

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants